Ambiguss, a game for building a Sense Annotated Corpus for French

Evaluating a WSD task is a challenge at least as difficult as developing the task itself. Manually constructing a corpus of ambiguous sentences is a difficult and tedious task. Such a corpus is even more complex to produce if in each sentence, each ambiguous word is associated with its correct meaning. In addition to finding/imagining the sentence, it is necessary to have a word sense lexicon to associate the correct meaning with each ambiguous word, and such a resource may not be available in any given language. Secondly, referring to a given meaning might be tricky. This can be done by associating a meaning number, but in such a case, the lexicon must be provided along with the corpus. Another method would be to represent the correct meaning of the ambiguous word by a gloss, i.e. a word or group of words that intuitively refers to its correct meaning. A classic example could be bank > river and bank > money; the glosses river and money refer to two possible meanings of the word bank. Anyway, to define the glosses, i.e. to choose the word that best illustrates the meaning of a word is another pitfall: Indeed, it seems that there is not always a strong agreement between people for such a task.